CLAIMS 

1. An image processing apparatus comprising: 

an image obtaining unit that obtains an image captured 
with an image sensor; and 
5 a defect information generating unit that generates 

defect information indicating a defect within the image having 
been obtained, based upon a value at a target pixel and an 
* average value of a plurality of pixel values corresponding 
to pixels present within a predetermined range containing the 
10 target pixel. 

2. An image processing apparatus according to claim 1, 
wherein: 

the defect information generating unit includes a 
15 relative ratio calculation unit that calculates a relative 
ratio of the value at the target pixel and the average value 
of the plurality of pixel values corresponding to the pixels 
present within the predetermined range containing the target 
pixel, and generates the defect information based upon the 
20 calculated relative ratio. 

3. An image processing apparatus according to claim 1 or 
claim 2, wherein: 

the defect information generating unit generates defect 
25 information for an area within the image, which satisfies a 
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predetermined condition . 



4 . An image processing apparatus according to any of claims 
1 through 3, further comprising: 
5 a correction unit that corrects the defect within the 

image based upon the defect information. 

5. An image processing apparatus according to claim 2, 
further comprising : 

10 a correction unit that corrects the defect within the 

image based upon the defect information, wherein: 

the correction unit corrects the defect by multiplying 
a value at a corresponding pixel by a reciprocal of the relative 
ratio . 

15 

6. An image processing apparatus according to claim 1, 
wherein: 

the image obtaining unit obtains a plurality of images 
captured with the image sensor; and 
20 the defect information generating unit generates defect 

information indicating a defect within one of the plurality 
of images by using the plurality of images having been obtained. 

7. An image processing apparatus according to claim 1, 
25 wherein: 
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the image obtaining unit obtains a plurality of images 
captured with the image sensor; and 

the defect information generating unit generates defect 
information corresponding to an entire image of each of the 
5 plurality of images by using the plurality of images having 
been obtained. 

8. An image processing apparatus comprising: 

an image obtaining unit that obtains a reference image 
10 photographed through an optical system; and 

a defect information generating unit that generates 
defect information indicating a defect within the reference 
image having been obtained, based upon a value of a target 
pixel and an average value of a plurality of pixel values 
15 corresponding to pixels present within a predetermined range 
containing the target pixel in the reference image. 

9. An image processing apparatus according to claim 8, 
wherein : 

20 the defect information generating unit includes a 

relative ratio calculation unit that calculates a relative 
ratio of the value at the target pixel and the average value 
of the plurality of pixel values corresponding to the pixels 
present within the predetermined range containing the target 

25 pixel, and generates the defect information based upon the 
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calculated relative ratio. 

10. An image processing apparatus according to claim 8 or 
claim 9, wherein: 

5 the image obtaining unit obtains a correction target 

image photographed through the optical system; and 

there is further provided a correction unit that corrects 
a defect within the correction target image based upon the 
defect information within the reference image. 

10 

11. An image processing apparatus according to claim 10, 
wherein: 

if the reference image and the correction target image 
have been photographed through an optical system in 
15 substantially identical optical conditions with regard to an 
aperture value and a pupil position, the correction unit 
corrects a value at a pixel constituting the correction target 
image by directly using the defect information having been 
generated. 

20 

12. An image processing apparatus according to claim 10, 
further comprising : 

a defect information conversion unit that converts the 
defect information in correspondence to at least either of 
25 an aperture value and a pupil position constituting optical 
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conditions of the optical system, wherein: 

if the reference image and the correction target image 
have been photographed through the optical system under 
different optical conditions with regard to at least either 
5 the aperture value or the pupil position, the correction unit 
corrects a value at a pixel constituting the correction target 
image by using the converted defect information. 

13. An image processing apparatus according to claim 9, 
10 further comprising: 

a correction unit that corrects a value of a 
corresponding pixel in the correction target image by 
multiplying the value of the corresponding pixel with a 
reciprocal of the relative ratio calculated for the reference 
15 image. 

14. An image processing apparatus according to claim 2 or 
claim 9, wherein: 

the relative ratio calculation unit sets the calculated 
20 relative ratio to 1 if the calculated relative ratio falls 
within a predetermined range containing 1. 

15. An image processing apparatus according to claim 14 , 
wherein : 

25 the relative ratio calculation unit correlates the 
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predetermined range over which the calculated relative ratio 
is set to 1 with a standard deviation value of the calculated 
relative ratio. 

5 16. An image processing apparatus according to claim 1 or 
claim 8, wherein: 

the predetermined range containing the target pixel is 
greater than a defect area manifesting within the image or 
the reference image. 

10 

17. An image processing apparatus according to claim 10, 
wherein: 

the image obtaining unit obtains a reference image 
photographed within a predetermined period of time preceding 
15 or following a time point at which the correction target image 
is photographed . 

18. An image processing apparatus according to claim 17, 
wherein: 

20 the image obtaining unit obtains a reference image 

photographed at a time point closest to or second closest to 
a time point at which the correction target image is 
photographed. 

25 19. An image processing apparatus comprising: 
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an image obtaining unit that obtains an image captured 
by using an image sensor capable of separating light into a 
plurality of colors; 

a luminance signal generating unit that generates a 
5 luminance signal based upon signals of the plurality of colors 
constituting the image ; and 

a defect information generating unit that generates 
defect information indicating a defect within the image based 
upon the luminance signal for the image having been generated. 

10 

20. An image processing apparatus according to claim 19, 
further comprising: 

a correction unit that corrects a value corresponding 
to a color component at a defective pixel within the image 
15 by using the defect information. 

21. An image processing apparatus according to claim 19, 
further comprising : 

a defect information generating unit that generates 
20 defect information indicating a defect within the image having 
been obtained based upon a value indicated by the luminance 
signal generated for a target pixel and an average value among 
values indicatedby luminance signals generated for a plurality 
of pixels within a predetermined range containing the target 
25 pixel. 
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22. An image processing apparatus according to claim 21, 
wherein : 

the defect information generating unit includes a 
5 relative ratio calculation unit that calculates a relative 
ratio of the value indicated by the luminance signal generated 
for the target pixel and the average value of the luminance 
signals generated for the plurality of pixels within the 
predetermined range containing the target pixel , and generates 
10 the defect information based upon the relative ratio having 
been calculated. 

23. An image processing apparatus according to claim 21, 
further comprising: 

15 a correction unit that corrects a value corresponding 

to a color component at a corresponding pixel by multiplying 
the value by a reciprocal of the relative ratio. 

24. An image processing apparatus according to claim 19, 
20 wherein: 

the image obtaining unit obtains a plurality of images 
photographed with the image sensor; 

the luminance signal generating unit generates the 
luminance signals for the plurality of images having been 
25 obtained; and 
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the defect information generating unit generates defect 
information indicating a defect within an image among the 
plurality of images by using the luminance signals generated 
for the plurality of images. 

5 

25. An image processing apparatus according to claim 19, 
wherein : 

the image obtaining unit obtains a plurality of images 
photographed with the image sensor; 
10 the luminance signal generating unit generates 

luminance signals for the plurality of images having been 
obtained; and 

the defect information generating unit generates defect 
information corresponding to an entire image of each of the 
15 plurality of images by using the plurality of images having 
been obtained. 

26. An image processing apparatus according to claim 1, 
wherein: 

20 the image sensor captures an image through an optical 

system; and 

the defect information is information on a projected 
image of a defect within an optical path, which manifests in 
the image. 

25 
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27. An image processing apparatus according to claim 26, 
wherein: 

the defect information generating unit simultaneously 
generates information indicating a position of the projected 
5 image of the defect within the optical path and information 
indicating intensity of the projected image of the defect 
within the optical path and records the position information 
and the intensity information. 

10 28. An image processing apparatus according to claim 27, 
wherein: 

the defect information generating unit moves the 
predetermined range over which the average value is calculated 
for each target pixel and generates continuous sets of 
15 information related to the intensity of the projected image 
of the defect within the optical path. 

29. An image processing apparatus according to claim 4, 
wherein : 

20 the correction unit determines a correction value by 

using an initial-signal value indicated at a correction target 
pixel position. 

30. An image processing apparatus according to claim 8, 
25 wherein: 
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the defect information is information on a projected 
image of a defect within an optical path, which manifests in 
the image. 

5 31* An image processing apparatus according to claim 30, 
wherein: 

the defect information generating unit simultaneously 
generates information indicating a position of the projected 
image of the defect within the optical path and information 
10 indicating intensity of the projected image of the defect 
within the optical path and records the position information 
and the intensity information. 

32. An image processing apparatus according to claim 31, 
15 wherein: 

the defect information generating unit moves the 
predetermined range over which the average value is calculated 
for each target pixel and generates continuous sets of 
information related to the intensity of the projected image 
20 of the defect within the optical path. 

33. An image processing apparatus according to claim 10, 
wherein : 

the correction unit determines a correction value by 
25 using an initial signal value indicated at a specific 
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correction target pixel position. 

34. An image processing apparatus according to claim 15, 
wherein: 

5 the relative ratio calculation unit sets the 

predetermined range over which the calculated relative ratio 
is set to 1 to a ± (3 x standard deviation value) range. 

35. An image processing apparatus according to claim 26, 
10 wherein: 

the predetermined range containing the target pixel is 
greater than a range over which the projected image of the 
defect within the optical path spreads inside the image. 

15 36. An image processing apparatus according to claim 30, 
wherein : 

the predetermined range containing the target pixel is 
greater than a range over which the projected image of the 
defect within the optical path spreads inside the reference 
20 image. 

37. An image processing apparatus comprising: 

an image obtaining unit that obtains a first image 
photographed through an optical system and a second image 
25 photographed under optical conditions different from optical 
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conditions in which the first image is photographed; and 

a defect information generating unit that generates 
defect information indicating a defect in the first image or 
the second image by using the first image and the second image . 

5 

38. An image processing apparatus according to claim 37, 
further comprising : 

a correction unit that corrects a defect in the first 
image or the second image by using the defect information. 

10 

39. An image processing apparatus according to claim 37, 
wherein: 

the first image and the second image are photographed 
under different optical conditions with regard to at least 
15 either of an aperture value and a pupil position. 

40 . An image processing apparatus according to any of claims 
37 through 39, wherein: 

the defect information generating unit includes an 
20 optical condition conversion unit that converts at least either 
the first image or the second image so as to conform to a specific 
optical condition, in order to eliminate a mismatch of the 
optical conditions for the first image and the second image. 

25 41. An image processing apparatus according to claim 40, 
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wherein : 

if the optical conditions with regard to the aperture 
value are different, the optical condition conversion unit 
executes low pass filter processing on a pixel signal generated 
5 based upon the first image or the second image so as to convert 
a defect state corresponding to the first image or the second 
image to a defect state estimated to manifest at a matching 
aperture value. 

10 42. An image processing apparatus according to claim 41, 
wherein: 

the optical condition conversion unit executes 
conversion by using a substantially uniformly weighted low 
pass filter. 

15 

43. An image processing apparatus according to claim 40, 
wherein: 

if the optical conditions with regard to the pupil 
position are different, the optical condition conversion unit 

20 executes displacement processing through which a pixel signal 
generated based upon the first image or the second image is 
displaced froma center of an optical axis of the optical system 
along a direction of a radius vector so as to covert a defect 
state corresponding to the first image or the second image 

25 to a defect state estimated to manifest at a matching pupil 
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position. 



44. An image processing apparatus according to claim 43, 
wherein: 

5 the optical condition conversion unit executes 

displacement processing through which a pixel signal located 
further away from the center of the optical axis is shifted 
to a greater extent along the radius vector. 

10 45. An image processing apparatus according to claim 43 or 

claim 44, wherein: 

the optical condition conversion unit executes the 

displacement processing by executing an arithmetic operation 

to predict an extent of displacement on an assumption that 
15 foreign matter causing the defect is present over a specific 

distance from an image-capturing surface within the optical 

system along the optical axis. 

46. An image processing apparatus according to claim 37, 
20 wherein: 

one of the first image and the second image is a correction 
target image to undergo correction and the other image is a 
reference image used to generate the defect information. 

25 47. An image processing apparatus according to claim 37 , 
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wherein: 

the first image and the second image are both correction 
target images to undergo correction; and 

the defect information generating unit generates defect 
5 information to be used commonly in conjunction with the first 
image and the second image by using the first image and the 
second image . 

48. An image processing apparatus according to claim 4 7 , 
10 wherein: 

the defect information generating unit includes an 
optical condition conversion unit that converts at least either 
the first image or the second image so as to conform to a specific 
optical condition, in order to eliminate a mismatch of the 
15 optical conditions for" the first image and the second image. 

49. An image processing apparatus according to claim 46, 
wherein : 

the image obtaining unit obtains the reference image 
20 photographed at an aperture value corresponding to a narrowest 
aperture opening setting in an adjustable aperture value range 
of the optical system. 

50. An image processing apparatus according to claim 37 , 
25 wherein: 
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the defect information generating unit generates defect 
information indicating a defect within the image having been 
obtained, based upon a value of a target pixel and an average 
value of a plurality of pixel values corresponding to pixels 
5 present within a predetermined range containing the target 
pixel in the image. 

.51. An image processing apparatus according to claim 4 6 or 
claim 49, wherein: 
10 the image obtaining unit obtains a reference image 

photographed within a predetermined period of time preceding 
or following a time point at which the correction target image 
is photographed. 

15 52. An image processing apparatus comprising: 

an image obtaining unit that obtains a first image 
photographed through an optical system and a second image 
photographed under optical conditions different from optical 
conditions in which the first image is photographed; and 

20 a correction unit that corrects a defect contained within 

the first image or the second image by using the first image 
and the second image. 

53. An image processing apparatus according to claim 52, 
25 wherein: 
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the first image and the second image are photographed 
under different optical conditions with regard to at least 
either of an aperture value and a pupil position. 



5 54. An image processing apparatus comprising: 

an image obtaining unit that obtains a photographic image 
captured with an image sensor; 

a flat portion extraction unit that extracts a flat 
portion area within the photographic image having been 
10 obtained; and 

a defect information generating unit that generates 
defect information corresponding to the extracted flat portion 
area . 

15 55. An image processing apparatus according to claim 54 , 
further comprising: 

a correction unit that corrects an image within the flat 
portion area based upon the defect information. 

20 56. An image processing apparatus according to claim 54 or 
claim 55, wherein: 

the defect information corresponding to the flat portion 
area is generated based upon a value at a target pixel and 
an average value of a plurality of pixel values corresponding 

25 to pixels present within a predetermined range containing the 
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target pixel in the image within the flat portion area. 

57. An image processing apparatus according to claim 56, 
wherein: 

5 the defect information generating unit includes a 

relative ratio calculation unit that calculates a relative 
ratio of the value at the target pixel and the average value 
of the plurality of pixel values corresponding to the pixels 
present within the predetermined range containing the target 
10 pixel, and generates the defect information corresponding to 
the flat portion area based upon the calculated relative ratio . 

58. An image processing apparatus according to claim 55, 
further comprising : 

15 a relative ratio calculation unit that calculates a 

relative ratio of a value at a target pixel and an average 
value of pixel values corresponding to a plurality of pixels 
present within a predetermined range containing the target 
pixel, among pixels constituting an image of the flat portion 

20 area, wherein: 

the defect information generating unit generates the 
defect information corresponding to the flat portion area based 
upon the relative ratio having been calculated; and 

the correction unit uses a reciprocal of the relative 

25 ratio corresponding to a pixel in the image of the flat portion 
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area when correcting a value of the corresponding pixel in 
the image of the flat portion area by multiplying the pixel 
value by the reciprocal . 

5 59. An image processing apparatus according to claim 58 , 
wherein: 

the correction unit executes low pass processing on the 
relative ratio which has been generated as the defect 
information and corrects the value of the corresponding pixel 
10 in the image of the flat portion area by multiplying the pixel 
value by a reciprocal of the relative ratio having undergone 
the low pass processing, which corresponds to the pixel in 
the image of the flat portion area. 

15 60 . An image processing apparatus according to any of claims 
54 through 59, wherein: 

the flat portion extraction unit executes edge 
extraction within the photographic image and extracts an area 
in which no edge is extracted as a flat portion area. 

20 

61. An image processing apparatus according to claim 54, 
wherein: 

the flat portion extraction unit includes a gradation 
conversion unit that executes gradation conversion on the 
25 photographic image and executes a flat portion area extraction 

102 



on the photographic image having undergone the gradation 
conversion. 

62. An image processing ' apparatus according to claim 61, 
wherein : 

when gradation of the photographic image is indicated 
with a linear signal, the gradation conversion unit converts 
the linear signal to a nonlinear signal. 

63. An image processing apparatus according to claim 62, 
wherein : 

the gradation conversion unit executes conversion by 
enlarging the gradation on a low intensity side and compressing 
the gradation on a high intensity side. 

64. An image processing apparatus according to claim 62 or 
claim 63, wherein: 

the gradation conversion unit executes conversion by 
using a power function. 

65. An image processing apparatus according to claim 64, 
wherein: 

the power function is a square root function. 

66 . An image processing apparatus according to any of claims 
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60 through 65, wherein: 

the edge extraction is executed by calculating 
differences corresponding to a plurality of distances between 
a target pixel and surrounding pixels along a plurality of 
5 directions. 

67 . An image processing apparatus according to any of claims 
55, 58 and 59, further comprising: 

a luminance level decision-making unit that makes a 
10 decision as to whether or not a luminance level of the 

photographic image is equal to or higher than a predetermined 
luminance level, wherein: 

the correction unit executes correction for an area 
determined to be a flat portion area, where the luminance level 
15 is equal to or greater than the predetermined level. 

68. An image processing apparatus according to claim 54, 
further comprising: 

a reference image obtaining unit that obtains a reference 
20 image captured with the image sensor; and 

a reference image defect information generating unit 
that generates defect information corresponding to the 
reference image, wherein: 

the defect information generating unit generates the 
25 defect information corresponding to the flat portion area by 
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using area information included in the defect information for 
the reference image and area information corresponding to the 
flat portion area in combination. 



5 69. An image processing apparatus according to claim 68, 
wherein : 

if an area that is not extracted as the flat portion 
area is still indicated to be a defect area by the defect 
information for the reference image, the flat portion 
10 extraction unit extracts the defect area as a flat portion 
area . 



70. An image processing apparatus according to claim 68 or 
claim 69, wherein: 
15 the defect information generating unit generates the 

defect information for an area indicated as a defect area by 
the defect information for the reference image and also 
determined to be the flat portion area. 

-20 71. An image processing apparatus according to claim 68, 
further comprising: 

a defect information conversion unit that converts the 
defect information for the reference image to defect 
information equivalent to defect information for a reference 
25 image photographed under optical conditions identical to 
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optical conditions under which the photographic image has been 
photographed when the photographic image and the reference 
image have been photographed under different optical 
conditions, wherein: 
5 the correction unit uses the defect information for the 

reference image resulting from the conversion. 

72. An image processing apparatus according to claim 69 or 
claim 70, further comprising: 

a defect information conversion unit that converts the 
defect information for the reference image to defect 
information equivalent to defect information for a reference 
image photographed under optical conditions identical to 
optical conditions under which the photographic image has been 
photographed when the photographic image and the reference 
image have been photographed under different optical 
conditions, wherein : 

the flat portion extraction unit and the correction unit 
use the defect information for the reference image resulting 
from the conversion. 

73. An image processing apparatus according to claim 71 or 
claim 72, wherein: 

in consideration of an error in defect information 
25 conversion executedby the defect information conversion unit, 
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the correction unit expands the defect area indicated by the 
defect information for the reference image, at least by an 
extent corresponding to the error in the defect information 
conversion. 

5 

74. An image processing apparatus according to claim 54 , 
wherein : 

the image obtaining unit obtains a plurality of 
photographic images captured with the image sensor; 
10 the flat portion extraction unit extracts the flat 

portion area in each of the plurality of photographic images; 
and 

the defect information generating unit generates defect 
information corresponding to the flat portion area in one of 
15 the plurality of images by using images of flat portion areas 
in the plurality of images having been extracted. 

75. An image processing apparatus according to claim 54, 
wherein : 

the image obtaining unit obtains a plurality of 
photographic images captured with the image sensor; 

the flat portion extraction unit extracts the flat 
portion area in each of the plurality of photographic images; 
and 

the defect information generating unit generates defect 
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information corresponding to an entire image of each of the 
plurality of images by using images of flat portion areas in 
the plurality of images having been extracted. 

5 76. An image processing apparatus according to claim 74 or 
claim 75, wherein: 

the defect information corresponding to the flat portion 
area is generated based upon a value at a target pixel and 
an average value of a plurality of pixel values corresponding 
10 to pixels present within a predetermined range containing the 
target pixel in the image within the flat portion area. 

77. An image processing apparatus according to claim 76, 
wherein: 

15 the defect information generating unit includes a 

relative ratio calculation unit that calculates a relative 
ratio of the value at the target pixel and the average value 
of the plurality of pixel values corresponding to the pixels 
present within the predetermined range containing the target 

20 pixel, and generates the defect information corresponding to 
the flat portion area based upon the calculated relative ratio . 

78. An image processing apparatus according to claim 69, 
wherein : 

25 if there are a predetermined number of pixels or more 
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pixels from which an edge has been extracted are present around 
the defect area indicated by the defect information for the 
reference image, the flat portion extraction unit does not 
extract the defect area as the flat portion area. 

5 

79. An image processing apparatus according to claim 78, 
wherein: 

if an edge has been extracted from a majority of pixels 
among pixels present in a predetermined area surrounding a 
10 pixel in the defective area, the flat portion extraction unit 
does not extract the pixel in the defect area as a pixel in 
the flat portion. 

80. A computer-readable computer program product having an 
15 image processing program enabling a computer to execute 

functions of an image processing apparatus according to any 
of claims 1 through 79. 
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